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[57] ABSTRACT 

A method and apparatus for identifying programs such 
as television programs received from various sources 
detects the occurrence of predetermined events such as 
scene changes In a video signal and extracts a signature 
from the video signal. The signatures and the times of 
occurrence of the signatures are stored and subse- 
quently compared with reference signatures to identify 
the program. The signatures may be compared in pairs 
to increase resolution, and the time interval between 
events or signatures may also be used to identify the 
program, either by themselves or in conjunction with 
the signatures. 

108 Claims, 7 Drawing Figures 
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continuously, and consequently, requires a large com- 

METHODS AND APPARATUS FOR puter to process the data thus generated. 

AUTOMAnCALLY IDENTIFyiNG PROGRAMS Another system that monitors the program content of 

VIEWED OR RECORDED a broadcast signal is disclosed in U.S. Pat. No. 4,230,990 

5 to Lert Jr. et al. The system disclosed in the Lert Jr. et 

BACKGROUND THE INVENTION al patent reduces the amount of data that must be pro- 

cessed, as compared to the amount of data processed by 
h Field of the Invention ^ , ^ the Moon et al, patent, by utilizing cues that are either 
This mvention relates generally to methods and appa- externally generated or present in the program material 
ratus for Identifying programs and.the viewing habits of jq to initiate the signature generation and correlation pro- 
the pubhc, and more particularly to a method and appa- ^^^^^ ^ j,^ ^een detected. However, the 
ratus for identifying programs, such as television pro- ^y^^^^ disclosed in the Lert Jr. et al. patent is designed 
grams obtained from various sources, including video monitor the programs broadcast by broadcasting 
recorders, stations such as network affiliated stations in order to 
2. Description of the Prior Art 15 determine whether they are broadcasting the programs 
Systems for identifying programs such as television required by the networks to be broadcast, rather than as 
programs that are broadcast or viewed are known. Such a system for determining the viewing habits of the pub- 
systems fall into various categories. These categories lie. Thus, many of the problems that occur when the 
include manual systems wherein the viewer keeps a habits of a television viewer are being monitored, such 
diary indicating all of the programs viewed; mechani- 20 as, for example, frequent channel changing, and the 
cal, electromechanical and electronic systems that sense problems that occur in identifying programs that are 
the channel to which a television receiver is tuned; recorded and subsequently played back by a home 
systems that detect identifying signals present in the video recorder are not addressed, 
television broadcast signal; and systems that monitor SUMMARY OF THE INVENTION 
and analyze the program content of the broadcast signal 25 SUMMARY Ut 1 Hb INVENTION 
utilizing image processing and correlation techniques to It is an object of the present invention to provide a 
identify the program. method and apparatus for identifying programs that 
While all of these systems do provide a way to iden- overcome many of the disadvantages of the prior art 
tify programs and to monitor the viewing habits of the systems. 

public, the manual methods are slow in acquiring data 30 It is another object of the p resen t invention_to_i}rot^ 

and are prone to inaccuracies resulting from the entry of vide an improved method and^parat^usTfOT deterra 

erroneous data that may be intentionally or unintention- Cingztheiyiewng-habits-of :the^ that utilizes -the::; 

ally entered. Systems that monitor the channel to which cicpntent^of Jhe-progiwn-being,viewed.-- 

a receiver is tuned overcome the drawbacks of the .^^ another object of the present myeji^^ 

manual systems, but require access to the tuning mecha- vide an improved mejhod^nda^^^^^^ 

nism of the receiver. Such access is becoming increas- broadcastsrand determmm^^^^ 

ingly difficult with the advent of cable and pay televi- <tions:are:b^ 

, ^ * v -* „ J J J messages that they are req uired to broadcasts 

sion systems that utilize vanous converters and decod- ^"iS^^-^-i ic — — ^""clZ * • 4 

P , . v j u * ^ \A It IS yet another object of the present invention to 

ers, many of which are difficult to access. Moreover, a ^ ^ a\ tA 

\_ : . - J * r 1 J r 40 provide a method and apparatus for identifying previ- 

such systems cannot identify programs played from a ^ , , , ««ru*:«« u« « 

^ , . -j j c *. ously recorded programs being viewed by a viewer, 

source such as a home video recorder. Systems tha j/j^ another object of the present invention to 

detect Identification signab encoded on the broadest ^ ^ that accurately identifies programs 

signal overcome iMny of the accuracy and access disad- ^^j^j, ^ ^j^^ ^^^^ computation utilizing data 

vantage described above, but require the cooperation correlation techniques on the program 

of the broadcasters who must encode the programs material 

prior to broadcast for such systems to be effective. ciris-yet:anothlF:5bjeiror:theTpre^^^^^ 

Consequently, such systems are useable only on a lun- pro^^dVaIprog^am-identiflcationrsl^tiS^that:ext^lc 

itcd number of broadcasts, such as network broadcasts. p-^SiFamridentifyingrsignatures-orUSOIpS^^ 

Systems that monitor and analyze the content of the 5^ rSrceiof certaifirpredSermiHed eve^^^^ 

program itself theoretically eliminate the disadvantages ^ Jt~isEaioth^bjea:of-lh"E:p^ to-proo 

of the other systems, but heretofore the information yjag^ -progranrjdentifying- system^ th'at extracts'pro-^ 

processing requirements of such systems rendered such gram identifyingTsigiiaturesJrom^ 

systems impractical or uneconomical because of the aiid-comparM:th'e:Mtracted::agn^^ 

computing capacity required to analyze the vast quan- 55 signatures in-pairs-to-provide m ore a ccumejd entifira^^ 

tity of information present in a broadcast signals. More- (^\onr^^ ^ 

over, previous attempts to reduce the quantity of infor- it is yet another object of the present invention to 

mation processed tended to introduce inaccuracies. provide a program identifying system that monitors the 

An example of a system that utilizes the program program material for the occurrence of predetermined 

content of a broadcast signal to identify the program is 60 events, and monitors the time interval between such 

described in U.S. Pat, No. 3,919,479 to Moon et al. The events to identify the program, utilizing the events ei- 

Moon et al. system utilizes a non-linear analog trans- ther by themselves or in conjunction with the extracted 

form to produce a low frequency envelope waveform, signatures. 

and the information in the low frequency envelope of a It is yet another object of the present invention to 

predetermined time interval is digitized to generate a 65 provide a system that alters the identification criteria 

signature. The signatures thus generated are compared depending on whether the program being viewed is a 

with reference signatures to identify the program. How- broadcast program of a program previously recorded 

ever, the Moon et al. system generates the signatures by the viewer. 
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Therefore, in accordance with a preferred embodi- 
ment of the invention, there is provided a system having 
a home unit that monitors the viewing conditions in the 
home, for example, whether the television set is on or 
off, whether any home video recorder (e.g., a video 5 
cassette recorder or VCR) is on or off and the video 
signal being received. By monitoring the aforemen- 
tioned signals and functions, the system determines 
whether no viewing is occurring, whether a television 
broadcast is being viewed, whether a television broad- 10 
cast is being recorded, whether viewing and recording 
is occurring simultaneously, or whether a previously 
recorded program is being viewed. Once the viewing 
mode has been established, .the program being" vie^x^'is^ 
[d^ntified^y^extractjng-aicharac^ 1 5 

tK^e:yideo:signaLand-storing:jt:for^bsequenr^ 
^on-viatlr 

Once theTnode of viewing has been determined, a 
signature of the program material being viewed or re- 
corded is extracted, provided certain predetermined 20 
events occur first. These events shall be referred to as 
Event 1 and Event 2. Several events can serve as Evenc 

1. These events may be a turn on of the television set, a 
channel change, a predetermined time interval, for ex- 
ample 5 to 10 minutes, since the occurrence of the previ- 25 
ous Event 1, or a sudden change to a black scene. 
Events that can serve as an Event 2 include a scene 
change to a scene other than a black scene, a color 
change, which may be an overall color change or a 
color change of a predetermined line, a still picture as 30 
evidenced by several consecutive substantially identical 
frames, an audio level change, and others. 

Upon the occurrence of an Event 1, the home unit 
monitors the broadcast for the occurrence of an Event 

2, and extracts a pair of signatures from the video signal 35 
immediately following the next two Event 2s following 
an Event 1. Or, if a previously recorded program is 
being viewed, the system extracts three or more signa- 
tures from the video signal immediately following the 
next three or more Event 2s following an Event I. The 40 
signatures are extracted by detecting and digitizing the 
envelope of the video signal immediately following the 
Event 2s. The digitized signatures, as well as the times 
that the signatures were extracted, are stored for future 
comparison with reference signals stored in a central 45 
location. The time intervals between Event 2s are also 
stored and used to identify the program cither by them- 
selves or in conjunction with extracted signatures. The 
time intervals between Event 2s arc particularly useful 
for identifying the playback of previously recorded 50 
video tapes for reasons that will be discussed in a subse- 
quent portion of the specification. 

Theiunit.uscd to:acquire~the-reference:signaturcs:that:i 
are^compared^ith^the-signatures-received-fronPt^ 
home-units-is-similarito:the homc-.unit-cxcept-that-the-55 
reference_signature^acq uisition— unit — monitors— each V 
broadcast^pntiniiously,-and:does:not:W 
1 to iiutiate a signature extraction. Therefore, each 
broadcast is continuously monitored and a sig nature is 
extracted each time an Event 2 is detected. The-signa--^0 
tures,':asiwell-as-thc times of-occun-en 
tures^are stored jo form- a library of "reference signa- 
tures^. The stOTcd^signatures are compared witli signa-- 
u^iobtainedcfroih^he-homeunitthat^occ^ 
appimimately:^the_same_time^^llic-ti^ be- 65 

twe en-Eve nt-2s-are-also^ stored ^for comparison with 
corresponding]!^ the Hbinc" 

units: ~~ ^ ^ 



209 

4 

DESCRIPTION OF THE DRAWINGS 

These and other objects and advantages of the pres- 
ent invention will become readily apparent upon con- 
sideration of the following detailed description ana 
attached, drawing wherein: 

FIG. 1, is a block diagram of the home unit of the 
program identification system according to the present 
invention; 

FIG. 2 is a block diagram of the central office and 
reference signature extraction portions of the system 
according to the invention; 

FIG. 3 is a chart illustrating the various modes of 
viewing and recording that can occur in a typical 
household; 

FIGS. 4 and 5 are flow charts illustrating the logical 
steps performed by the home and central office units, 
respectively; 

FIG. 6 is a flow chart illustrating the logical steps 
performed by the central office unit in identifying signa- 
tures; and 

FIG. 7 illustrates how signatures and scene changes 
may be extracted from the video envelope. 

DETAILED DESCRIPTION OF THE 
PREFERRED EMBODIMENT 

Referring now to the drawing, with particular atten- 
tion to FIG. 1, there is illustrated a block diagram of the 
home unit of the system according to the invention 
generally designated by the reference numeral 10. The 
system 10 receives signals to be identified from a televi- 
sion receiver 12, and in many instances from a home 
video recorder, such as a video cassette recorder or a 
VCR 14, In a typical home installation, the receiver 12 
and the VCR 14 receive signal from a receiving antenna 
16, and in many instances from a cable television system 
18. These signals may be applied directly to the receiver 
12 and to the VCR 14, but it is convenient to employ a 
switching network 20 that selectively switches the re- 
ceiver 12 and VCR 14 to the antenna 16 and to the cable 
18 in order to permit the viewing and recording of 
broadcasts received from the antenna 16 or from the 
cable 18. In addition, the switching network 20 may 
connect the VCR 14 to the receiver 12 to permit prere- 
corded or previously recorded tapes to be played back 
by the VCR and viewed by the receiver 12. 

Video signals representative of the program applied 
to the receiver 12, as well as video signals representa- 
tive of the signals being recorded by the VCR 14, are 
applied to a video processing circuit 22, for example, by 
a video switching circuit 24 which permits either the 
receiver 12 or the VCR 14, or both, to be monitored. 
The video processing circuit 22 includes a detector and 
a low pass filter (discussed subsequentiy in greater de- 
tail), and provides a signal representative of the enve- 
lope of the video signal to an analog-to-digital con- 
verter 25 which generates a digital representation of the 
video envelope. The digitized video envelope signal is 
applied to an events detector 26 which detects predeter- 
mined events that occur in the digitized video signal, 
and causes the signature of^he digitized video signal to 
be extracted by signature extraction circuitry 28 upon 
the occurrence of a predetermined sequence of events. 
The signature thus generated is stored in a data storage 
system 30 along with the time that the signature was 
extracted. The extraction times are provided by a clock 
32 which provides a digital representation of time to the 
data storage system 30 via, for example, a combining 
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circuit 34. The clock 32 may be a real time clock or a recorder. Modes 4.0, 4,3 and 4.4 can be detected by 

relative time clock that is periodically reset from a cen- logic circuitry. 

tral location. The function of the events detector 26, the As previously stated, the home unit 10 monitors the 

signature extractor circuitry 28, the data storage 30 and viewing mode and the video signal, and extracts signa- 

the clock 32 can be performed by individual circuits as 5 tures from the video signal only if certain events occur. 

shown or by a microprocessor based^stem^ ^ ^ The logic necessary to provide this function is illus- 

The home unit 10 isjnten-ogated at penodiainterval^^ trated in FIG, 4. As is apparent from FIG. 4, the mode 

^for cxMplcronce a dayrby^axen^^ of operation is recorded whenever there is a change in 

ally^designated^y^j^ere^ numera]:40:(FIG. 2), the mode, such as, for example, a change from viewing 

which compaTO-the signaturcs-from: tficiva rious-Kome^lO to recording, etc. Each time the mode changes, a deter- 

. unitsiwith:reference:signatures:in:order-to identify-the:> mination is made in order to determine what the new 

signatures'jronrthcjhomci^ The home units may be ^^ode is. This is accomplished by determining whether 

periodically interrogated by a data communications the television is on or off, whether the VCR is on or off, 

circuit 42 that accesses the vanous home units via a ^^6 whether the VCR is in a record or playback mode, 

suitable communications system, jorjxampje^pk^^ Once:it:has been-detcffliHed:whTth"^r^^ 

n ity of telephone lines 44. The.signatures thus collected-- yeR-areWor ofrrand whetheiuhe-VGR is in a record^ 

I are,compare£by-a-^nn:al^^ or-playback-mode,-the mode can readily be determined:::. 

\\si^tures-stored-^m a-data-basel46.-^C^todic b^ing^look-upitableTcohtmmi^^ 

\ systeml4«-controlsithe:collec FKKlrSTdJh^inodirirr^ ~~ 

\ the^signaturesj-ec^^ 20 ^^e mode is determined to be Mode 1, indicating no 

the^generationqf^fe^ viewing, no further action is taken untU the mode 

: data^b^46. The-latter funct^ 1^ determined that the mode of oper- 

junction jyith:a:plurahty^o^cfcr^^ ^^-^^ ^^^^^^ ^^^^ 2 ^^^^ ^ ^^^^^^^ 

circmts.SO that arej^ television receiver for the occurrence of an Event 1, 

The reference signature extraction circuits monitor the 25 ^^.^^ 

any one of the followine: 

vanous networks, cables and other signal sources in j television turn-on 

those cities, and extract reference signals, for example, ' ^u^nTiel chanee 

whenever an Event 2 occurs in any of the signals being * ^ ;«f-...<»i « <; m •v.i^nf^e^ 

•* J in- r ^- i„ 3. a predetermined time mterval (e.g 5-10 minutes) 

monitored. The reference signals thus extracted are .1; . „^^,,:„„c Pv,p„t i 
transmitted to the central office unit 40 and stored in the 30 , smce ine previous tveni i 
data base 46 along with the times that such signatures ^' f ^"^^en scene change to a black scene, 
are extracted, as provided by a clock 52. The clock 52 5. loss of synchronization for an appreciable penod of 
of FIG. 2 is similar to the clock 32 of the home unit and „ , , , . 
serves to indicate the time of occurrence of the refer- S|ich Event Is are relatively easy to detect For exam- 
ence signatures. The clock 52 may also be a real time 35 P'^r television turn on can be detected simply by mom. 
clock or a relative time clock that is periodically set by appropriate voltage or current. Television 
the central office 40. A central reference signature ex- ^um-on is not used as an Event 1 m the system lUus- 
traction circuit 54 and associated clock 56 serve to ex- t^^^ed m FIG. 4 but may be used in other systems, par- 
tract signatures and times of extraction of cable origi- t^^^l^^y systems that do not monitor a VCR and conse- 
nated programs and signatures of prerecorded pro- 40 qucntly need not employ mode logic. A channel change 
grams. Altemativelyr«tr^ti6ircircuits:may,be placed, ^^n be detected by monitonng the position of the tuner 
^tithe-head:endsiof:cable-system-to^«^Sict^fe mechanism when mechanical tuners are used, by moni- 
I signatures:of:cable:programs: The:signature:extractionr:> tonng tuning voltage m electrically tuned tuners, by 
, perf6ffled-by.jhe::yariou^ monitoring local oscillator frequency, or even more 
\\ storage is controlled by the cOTtTil^BSSprniP^t^ simply by monitonng the video synchronization pulses, 
""ThThomTiSiFof the system according to the inven- preferably the vertical syncronization pulses and mdi- 
tion monitors the mode ofviewing by monitoring the on eating a channel change in the event of a loss or a 
and off and other functions of the television receiver change in the syncronization pulse for a short time 
and any video recorder that may be used in the home. interval. The predetermined time mterval can readily be 
The various modes of viewing or recording are illus- 50 determined from a clock, and a sudden scene change to 
trated in FIG. 3. Mode 1 occurs when both the VCR black can be readily detected by monitonng the average 
and television receiver arc both off. and indicates that value of the video signal and indicating a change to a 
no viewing is taking place. Mode 2 represents television black scene when the average value becomes that repre- 
viewing and occurs when the television receiver is on sentative of an all black scene. A loss of synchronization 
and the VCR is off. Mode 3 represents recording by the 55 for an appreciable length of time Gongcr than that 
home VCR and occurs when the television receiver is caused by a channel change) indicates fast forward or 
off and the VCR is on. Mode 4 occurs when the VCR rewind mode of operation of a VCR (Mode 4.3 or 4.4) 
and the television receiver are both on, and can repre- or the loss of the video signal (e.g. station off the air), 
sent one of three conditions. The first condition, desig- As long as no Event 1 occurs, the home unit records 
nated as mode 4.0, represents televbion viewing 60 no new data; however, upon the occurrence of an 
through the tuner of the VCR without recording. Mode Event I, the system is conditioned to extract a signature 
4.1 represents VCR recording and television viewing, upon the occurrence of the next two consecutive Event 
while the third condition, designated as Mode 4.2, rep- 2s, with an Event 2 being defined as: 
resents VCR playback. Other modes of operation of the 1 . a scene change to a scene other than a black scene 
VCR, such as fast forward and rewind, are designated 65 2. a color change in successive frames or portions 
as Modes 4.3 and 4.4, respectively. Most of these modes thereof 
can be easily detected by monitoring power line voltage 3. a still picture 
or voltages elsewhere in the television set and the video 4. an audio level change. 
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Ways of detecting the various Event 2s described home units. The time intervals between Event 2s may 
above, as well as ways for extracting signatures will be be recorded or obtained from the times of occurrence oi" 

discussed in a subsequent portion of the specification. the Event 2s. 

Upon the occurrence of an Event 1, a counter is set to The logic employed by the central computer system 
zero and the video signal is monitored to determine if 5 48 of the central unit 40 in order to identify a program 

the television synchronization or sync signal is stable. is illustrated in FIG. 6. As is illustrated in FIG. 6, two 

This is accomplished, for example, by monitoring the consecutive signatures from a home unit are selected. If 

vertical synchronization pulses and waiting until a pre- there is not enough data to define two consecutive sig- 

determined number, for example, 16, consecutive stable natures, the next home unit is monitored. After two 

vertical sync pulses are detected before the unit is per- 10 consecutive signatures are selected, a check is made to 

mitted to detect an Event 2. If the synchronization has determine whether the signatures really are a pair, tor 

not been stable for a significant period of time, more example, that there is no Event 1 between the two sig- 

than five seconds, this period will be marked either as natures. 

television station is off the air or VCR fast forward or If the two signatures are a pair, the time associated 

rewind. 15 with the first home unit signature is read. All reference 

Once the sync has been stable for the required num- unit signatures in the data base 46 that occurred within 
ber of pulses, the system monitors the video signal for a predetermined time interval of the occurrence of the 
the occurrence of an Event 2. Whenever such an Event first home unit signature, for example, plus or minus 
2 change is detected, the time of the occurrence of the eight seconds, are selected for comparison, and the 
Event 2 is recorded as is the time interval since the 20 correlation coefficient between the first home unit sig- 
previous Event 2, if the time intervals between Event 2s nature and the selected reference unit signatures are 
are to be used as a means of program identification. The computed. If one of the correlation coefficient exceeds 
signature of the first frame following the last detected a predetermined threshhold, the time associated with 
Event 2 change is then extracted. Since a standard the second home unit signature is read. The next sev- 
broadcast television frame consists of two interlaced 25 era), for example, six, reference unit signatures thai 
fields, and since the two fields that form a frame are occur, for example, within a predetermined time inter- 
quite similar, it is not necessary to analyze both fields of val, for example, plus or minus eight seconds of the time 
the frame to obtain a signature. Typically, a satisfactory associated with the second home unit signature are 
signature is obtained from only a single field, for exam- selected, and the correlation coefficients between the 
pie, the first field of a frame. The method of extracting 30 second home unit signature and the reference unit signa- 
the signature will be discussed in greater detail in a tures are computed. If one of the correlations coeffici- 
subsequent portion of the detailed description, as will ent exceeds the predetermined threshhold, an identifica- 
the method of detecting an Event 2. tion is made and stored, and printed out, if necessary. 

Once the signature has been extracted, the counter is As discussed above, the system according to the in- 
incremented by 1. If the count in the counter is less than 35 vention utilizes a time reference to access reference 
two, indicating that more signatures are needed, the signatures that occurred at approximately the same time 
system is conditioned to respond to another Event 2 to as the home unit signature being evaluated in order to 
extract and record another signature. If the count in the eliminate the need for comparing the home unit sjgna- 
counter is two (or the required number of signatures) or ture with all of the reference signatures in the data base 
greater, indicating that the signatures required have 40 46. However, when a previously recorded program is 
been extracted, the system is conditioned to require an being played back, the time that the signature occurs 
Event 1 to occur before it responds to any further during playback cannot be used to locate the reference 
Event 2s. signal. Thus, in accordance with another important 

If it is determined that the mode of operation is Mode aspect of the present invention, the times of occurrence 
3, that is, VCR recording, it is not necessary for an 45 of signatures that occur when programs are being re- 
Event 1 to occur before the signal is monitored for corded are also stored. The times of occurrence of sig- 
Event 2s. The only determination that is made is natures that occur in Mode 3 (recording) are kept in a 
whether the synchronization is stable, and if so, the separate recording time file in the data base 46. Simi- 
system looks for Event 2s continuously and extracts the larly, in Mode 4.1 (viewing and recording) the times of 
signature and records the time of occurrence of the 50 signatures are also kept in the recording time file. Thus, 
signature each time an Event 2 is detected. Thus, all when the broadcast is played back (Mode 4.2), even 
signatures following every Event 2 are extracted and though the recorded signatures cannot be matched to 
stored so that upon subsequent playback, the program signatures stored in the reference unit during playback, 
may be identified even though only portions of the tape the signature can still be identified by utilizing the re- 
may be played back. 55 cording time instead of the playback time to retrieve the 

The logic for extracting reference signatures is illus- corresponding reference signatures from the reference 

trated in FIG, 5. Each broadcast and cable channel is signature data base 46. Thus, by keeping signatures 

monitored continuously by the reference signature ex- relating to programs that were broadcast several days 

traction circuits 50 and 54, and a signature is extracted or weeks ago within the data base 46, such previously 

each time an Event 2 occurs to provide a reference 60 recorded and subsequently played back broadcasts can 

library of reference signals. Consequently, since it is be readily identified. 

desired to extract a signature that occurs after every Alternatively, in order to identify the playback of 

Event 2, Event Is are not monitored. All that is re- previously recorded programs, instead of comparing 

quired is that the sync be stable and that an Event 2 the signature obtained during playback with reference 

occur. Thus, whenever an Event 2 occurs, the time of 65 signatures that represent the programs broadcast sev- 

occurrence of the Event 2 as well as the signature of the eral days or weeks ago stored in the recording time file 

scene following the scene change is recorded for future of the reference signature data base 46 (FIG. 2) of the 

comparison with signatures received from the various central office unit, the played back signatures may be 
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compared with signatures obtained by the home unit faded out. Because in a fade type scene change, the 

previously during VCR recording (Mode 3) and stored change occurs gradual! y» there is no drastic change in 

in the data storage 30 (FIG. 1) of the home unit. If the number associated with consecutive frames, and 

desired, the signatures may also be sent to the central consequently, such a scene change cannot readily be 

office, as are the signatures obtained during normal 5 detected by a system that compares immediately succes- 

television viewing, but these signatures would be la- sive frames. Numbers that are substantially identical for 

beled as VCR recording, and may be stored in the re- several consecutive frames may be used to indicated a 

cording time file. However, when playback occurs, the still picture. 

played back signatures would be compared with the Other methods of detecting an Event 2 include de- 
signatures stored in the home unit data storage. Only if 10 tccting a color change, for example, by detecting the 
a match occurred would the signatures be sent to the average amplitude of a particular color component, 
central office and compared with the signatures stored such as, for example, a red component, of each frame or 
in the central office data base 46 to identify the pro- ^ particular line or lines of each frame, and making 
gram. If no match occurred, the signatures would not comparisons of the types described above. Altema- 
be sent. This is because whatever is being played back 15 lively, different color components or an average of the 
cannot be identified, and may be somethmg like, for three color signals can be monitored for each frame and 
example, a home movie. Thus, the system eliminates the comparisons made. In addition, rather than monitoring 
need to search the entire recording file of the data b^e ^ ^^^5,^ f.^^^ ^^jy portions of a frame may be moni- 
46 in an attempt to identify something that cannot be ^^^^^ ^h^^. portions may be selected to be those por- 
identified. , t tions of a frame that normally do not change unless a 

In the playback mode, when attempting to match the ^^^^^ ^^^^^ ^^^^ ^^^^^^ ^^^^^ ^^^^ 

signatures obtained during playback with the signatures ^^^^^^ ^^^^^ ^^^^^^^ ^^^y^ ^^^^ ,^^,3 

stored in the data storage 30. it is convenient to use the jj^^, ^^^^^^^ ^ ^^^^^ movement occurring 

time intervals between the occurrence of the signature ^ ^^^^^ ^^^^^ 2s may be detected electron!- 

or Event 2s) to generate a time mterval signature, TIS 25 nionitoring the video envelope, or by monitor- 

(discussed m greater detail in a subsequent portion of ^ example, by placing a light 

the specification) to obtain the match. When^"^^^^^^ sensitive photoelectric device 36 (FIG. 1) in proximity 

time mterval signature for matchmg, three or more . , .u^\.^^Ur.^^ .^.^ tt,K* «,-^flr«Ki« L ^.J! ^r.r»Jr 

• * * »j zLj With the cathode ray tube, preterably in one corner 

consecutive signatures are extracted, and the time inter- . r aw *: 1 *i. j- 1 u 

vaJs between ^signatures are compared with the time 30 ^"f- Alternatively, the audio signal may be mom- 

intervals between signatures stored in the data storage processing circuit 38 and applied to 

30. When a pair of consecutive time intervals is found, ?^ analog-to-digital converter 25 to generate an Event 

a match is indicated. If greater accuracy is desired, 2 when the audio signal departs from a predetermined 

more than three consecutive signatures may be ex- level. . , , . ... 

tracted to provide more than two consecutive time 35 In accordance with an alternative embodiment of the 

intervals present invention that is particularly suitable for identi- 

Event 2s may be detected in a variety of ways. For fyi^S prerecorded programs, the time intervals between 
example, the video signal may be low pass filtered, for events, or m changes in one or more predetermined 
example, through a low pass filter having a cut-off fre- parameters, can be used to define a feature stnng. Such 
qucncy on the order of approximately 2 to 6 kiloHertz, 40 ^ system can be used either alone, or m combmation 
and its average amplitude detected. The signal may be with a signature extracting system to identify the pro- 
digitized by assigning a number representative of the S^am. When used alone, the amount of data generated 
average amplitude of the video signal for each frame. system is less than that generated by a signature 
•The numbers thus obtained may be compared in various generating system, and therefore, does not require 
ways. For example, the numbers obtained from differ- 45 ni"ch computation time to obtam a correlation. Conse- 
ent frames may be compared, and if the numbers are quently, it is not necessary to store the time of occur- 
substantially different, a scene change will be indicated. ^ence of each change (unless it is desired for other rea- 
The numbers compared may be obtained from consecu- sons), but rather, only the elapsed time between such 
tive frames or from non-consecutive frames,-for exam- changes need be stored. Such a system is particularly 
pie, frames spaced by a 20 to 40 frame interval, depend- 50 useful in identifying the playback of prerecorded pro- 
ing on the type of event that has been defined as the grams, or delayed broadcast programs because real time 
Event 2, Alternatively, a moving average of the num- data is not necessary to make the identification, 
bers representative of the video amplitudes of the last I" a system wherein the intervals between predeter- 
several frames may be calculated, and the number rep- mined changes in predetermined events are used to 
resentative of the average amplitude of the last detected 55 generate a time interval signature (TIS) the tiine inter- 
frame compared with the moving average to indicate a val signature, which can be represented as a digital code 
scene change if the number associated with the last sequence, can be constructed by measuring the time 
frame substantially differs from the moving average. mtervals between the sequential occurrences of regu- 

The various comparing schemes are useful for identi- larly occurring events in a broadcast Preferably, the 

fying various different types of Event 2s. For example, 60 chosen events should have the following properties: 

comparing the number associated with a particular I. Be readily measurable automatically, 

frame with the number associated with a previous frame 2. Occur at a sufficiently high rate that enough inter- 

is useful for detecting an Event 2 that may be termed an vals arc available to allow unique identification of 

instant scene change. A moving average comparison or the signal, 

a comparison with frames that occurred several frames 65 3. Always be present 

ago is useful for detecting an Event 2 that may be 4. Cannot be controlled by program originators to 

termed a fade type scene change wherein one scene is render identification vulnerable to changes in pro- 

gradually faded in and the previous scene is gradually gramming methods. 
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Typical events that can be used as the events to gen- 
erate a time interval string includes scene changes to 
black, Event 2s, including shifts in the average ampli- 
tude of the video signal, departures in the amplitude of 
the instantaneous video signal from the average, shifts 5 
in the video color balance, changes in the average color 
of a predetermined line, for example, a line near the top 
of the screen, a predetermined audio power level, for 
example, silence, etc. Thus, to define a time interval 
string, the home unit would a generate time interval *0 
string (TIS) by detecting one of the predetermined 
events, for example^ an Event 2 such as a scene change, 
and store the time intervals between consecutive 
changes. 

The monitoring of events can be done continuously 
to generate a long TIS without requiring an excessive 
memory capacity because such changes occur relatively 
infrequently, and the time intervals therebetween can be 
defined by only a few bits. The TIS's thus obtained 
from the home unit are compared with reference TIS*s 
stored in the central unit. 

However, in order further to reduce the amount of 
data that must be stored, the time intervals between the 
events being monitored may be monitored only after the 
occurrence of another predetermined event, such as an 
Event 1; and it has been found that satisfactory identifi- 
cation can be obtained by generating a feature string 
consisting of three time intervals between changes in 
the monitored variable, such as a scene change or other jq 
Event 2. The time interval strings thus generated are 
matched with reference time interval strings that are 
obtained in much the same way as those generated by 
the home units except that the reference time interval 
strings are continuously generated for each broadcast or 35 
other program source being monitored. 

Referring now to FJG. 7, there is shown a video 
envelope 100 as provided from the video processing 
circuit 22 (FIG. 1) to the analog-to-digital converter 25. 
The analog-to-digital converter 25 takes a plurality of 40 
samples 102 (FIG. 7) for example, 80 samples during a 
single frame interval, and generates a digital value rep- 
resentative of the amplitude of each of the samples 102. 
The digital values thus generated are applied to the 
events detector 26 (FIG. 1) and to the signature extrac- 45 
tion circuit 28 and used to detect the various events and 
to extract signatures. 

In order to detect an 'Event 2, for example, a scene 
change, the events detector 26 compares each of the 
digitized samples 102 of a current frame with a corre- 50 
sponding digitized sample 102 of a previous frame, ei- 
ther the immediately preceding frame or another previ- 
ous frame, depending on the type of scene change to be 
detected. The counterpart samples are then compared 
to determine whether a scene change has occurred. 55 
More specifically, the detection of a scene change oc- 
curs as follows: 

1. Take, for example, 80 samples of the video enve- 
lope of each frame 

2. Compare counterpart samples of a current frame 60 
and a previous frame 

3. Take the absolute value of the difference of each of 
the counterpart samples 

4. Take the average of the absolute values 

5. Indicate a scene change if the average of the abso- 65 
lute values exceeds a predetermined threshhold. 

The indication of a scene change is applied to the signa- 
ture extraction circuit 28 (FIG. 1) to cause the signature 
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extraction circuit 28 to operate on the digitized samples 
102 to extract the signature. 

The signature extraction circuit 28 operates on the 
digitized samples 102 by taking the average value 01* 
consecutive samples 102, for example, eight consecutive 
samples to generate a total of, for example, ten features 
104 that comprise an image signature. Typically, four 
bits define a feature, and consequently, an image signa- 
ture can be defined by forty bits of data for the example 
discussed above; however, in some instances more fea- 
tures and more data may be required. 

Both the home unit 10 (FIG. 1) and the reference 
signature extraction circuits 50 and 54 (FIG. 2) operate 
in the same manner to extract signatures, the only differ- 
ence in operation being that the signature extraction 
circuit 28 (FIG. 1) requires an Event 2 to be preceded 
by an Event 1 before signature extraction occurs; but 
the reference signature extraction circuits 50 and 54 
(FIG. 2) do not require an Event 1 to precede an Event 
2, but rather extract a reference signature each time an 
Event 2 occurs. 

Once the home unit signatures and the reference 
signatures are obtained, the home unit signatures will be 
compared with the reference unit signatures, normally 
with those signatures occurring in corresponding time 
intervals. The correlation coefficients between the 
home unit signatures and the reference unit signatures 
will be compared, and the reference signature that has 
the best correlation to the home unit signature will be 
used to identify the program viewed, provided that the 
correlation coefficient exceeds a minimum threshhold. 
If it does not, no match will be indicated. 

Obviously, modifications and variations of the pres- 
ent invention are possible in light of the above teach- 
ings. Thus, it is to be understood that, within the scope 
of the appended claims, the invention may be practiced 
otherwise than as specifically described above. 

What is claimed and desired to be secured by Letters 
Patent of the United States is: 

1. A method of identifying a signal received by an 
audience member's television receiver for determining 
audience ratings comprising the steps of: 

detecting the occurence of a first event in the func- 
tional operations of he audience namber's televison 
receiver; 

detecting the occurence of a second event after the 

detected first event in the signal to be identified; 
exti'acting a signature from a single frame of the video 

signal to be identified after the occurrence of the 

second event; 
storing the signature and the time of occurence 

thereof; and 

comparing the stored signature with reference signa- 
tures that occurred at approximately the same nme 
as the stored signature. 

2. The method recited in claim 1 wherein said signal 
is a television video signal and said second event is a 
scene change from one video scene to another video 
scene. 

3. The method recited in claim 1 wherein said signal 
is a color television video signal and said second event 
is a change in the color of at least a portion of successive 
frames. 

4. The method recited in claim 1 wherein said signal 
is a television video signal and said second event is a still 
picture. 
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5. The method recited in claim 1 wherein said signal means responsive to the detected first event for de- 
is an audio signal said second event is a predetermined tecting the occurrence of a second event in the signal to 
change in the magnitude in the audio signal. be identified; 

6. The method recited in claim 1 wherein said signal means for extracting a signature from a single frame 
is a television signal received by a television signal re- 5 of the video signal to be identified after the occu- 
ceiver and wherein said first event is the turn-on of the rence of the second event; 

televison receiver. means for storing the signature and the time of occu- 

7. The method recited in claim 1 wherein said signal rence thereof; and 

is a television signal received by a televsion signal re- ">cans for companng the stored signature with rcfer- 

ceiver receiver and wherein said first event is a change >0 ence signatures that occur at approximately the same 

in the channnel to which the television signal receiver is signature. . 

jyj^gj 16. The system recited in claim 15 wherein said signal 

8. The method recited in claim 1 whcnrin said first » t=»evision video signal and said second event detect- 
event is the elapse of a predetermined length of time „ '"8 ™""' "^"f^ for detecting a scene change 

*^ r-* * 15 from one video scene to another video scene, 

after the occurrence of the previous first event. . cx,o«^«, «o ,^«it^ ;« ik ...u^^^i^ c^;^ ♦^i*. 

„ , , . J • 1 - « . • -J J 17. A system as recited in claim lo wherein saio tele- 

9. The method recited in claim 8 wherein said prede- ^^^.^^ ^.^^^ ^ ^^^^.^^^ ^ ^^^^^^.^^ j 

termined length of time is approximately 5 to 10 mm- ^j,^^^.^ ^^.^ ^^^^ ^^^^^^^^ ^^^^^ ^^^^^ 

^^^^J. ^ -J. t includes means for detecting the turn-on of the tele vi- 

10. The method recited in claim 1 wherein said signal sion receiver 

is a television signal and said step of extracting said jg ^ ^yg^g^ recited in claim 15 wherein said signal 
signature includes the step of low pass filtering a video ^ television signal received by a television signal re- 
signal of the televison signal to provide a video envc- reiver, and wherein said first event detecting means 
lope signal, penodically sampling said video envelope includes means for detecting a change in the channel to 
signal to provide a series of video samples, and digitiz- 25 which the television signal receiver is tuned, 
ing said video samples to provide said signature. 19. a system as recited in claim 15 wherein said first 

11. The method recited in claim 10 wherein said step event detecting means includes means for detecting the 
of low pass filtering said video signal includes the step elapse of a predetermined length of time after the occur- 
of passing the video signal through a low pass filter rcnce of the previous first event. 

having a cut-off frequency of approximately 2 to 6 kHz. 30 20. A system as recited in claim 19 wherein said de- 

12. The method recited in claim 11 wherein the step tecting means includes means for setting the predeter- 
of sampling said video envelope signal includes the step mined length of time to approximately 5 to 10 minutes, 
of taking dghtly samples of said video envelope signal 21. A system as recited in claim 15 wherein said signal 
and taking the average of every eight consecutive sam- is a television signal and said means for extracting said 
pies to generate ten features that comprise the feature 35 signature includes means for low pass filtering a video 
string. signal of the television signal to provide a video enve- 

13. The method recited in claim 10 wherein the step lope signal, means for periodically sampling said video 
of sampling said video envelope signal includes the step envelope signal to provide a series of video samples, and 
of taking a predetermined number of samples of said means for digitizing said video samples to provide said 
video envelope signal and taking the averages of prede- ^ signature. 

termined consecutive samples to generate a plurality of 22. A system as recited in claim 21 wherein said low 

average signals, each average signal defining one of the Pass filtering means has a cut-off on the order frequency 

features that comprises the feature string. approximately 2 to 6 kHz. ^ . 

U. A system for identifying program material re- 23 A system as recited m claim 22 wherem said video 

ceived by an audience member^s televison receiver for ^5 envelope signal samphng means includ^ means for 

determining audience ratings comprising: ukingapproximately eighty sampl^ of smd videos 

means for detecting a first predetermined event in the lope signal and means for taking the average o^very 

functional operations of the audience member's televi- consecu we samples to generate ten features that 

compnse the feature string. 

sion receiver; j j . • j ^ . 50 24. A system as recited in claim 21 wherein said video 

means for detecting a second predetermined event m envelope signal sampling means includes means for 

the program material; . - . ^ taking a predetermined number of samples of video 

means for extracting at least one predetermined char- envelope signal and means for taking the averages of 

actenstic of the program matenal from a single frame of predetermined consecutive samples to generate a plu- 

a video signal; 55 rality of average signals, each average signal defining 

means for stonng the predetermined charactenstic features that comprises the feature string, 
and the time that the predetermined characteristic is 25. A system for identifying program material re- 
extracted; and ceived by an audience member's television receiver for 

means for comparing the recorded predtermined determining audience ratings comprising: 

characteristic and its time of occurrence with reference ^ means for detecting a first predetermined event in the 

predetermined characteristics that occured at substan- functional operations of the audience member's 

tially the same time to identify the program. television receiver; 

15. A system for identifying a signal received by an means responsive to the detected first predetermined 

audience member's television receiver for determining event for detecting at least one second predeter- 

audience ratings comprising: 65 mined event in the programmed material; 

means for detecting the occurrence of a first event in means for extracting at least one predetermined char- 

the functional operations of the audience member's acteristic of the program material from a single 

television receiver; ft-ame of a video signal after the occurrence of a 
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predetermined number of said first and second 
events; and 

means for recording the predetermined characteristic 
and the time of occurrence of the predetermined 
characteristic. 

26. The system recited in claim 25 wherein said sys- 
tem further includes a central unit, means for transmit- 
ting the recorded predetermined characteristics and 
times to the central unit, wherein the central unit in- 
cludes means for comparing the recorded predeter- 
mined characteristics and their times of occurrence 
with reference predetermined characteristics that oc- 
curred at substantially the same times to identify the 
program. 

27. The system recited in claim 26 further including 
means for generating said reference predetermined 
characteristics, wherein said reference predetermined 
characteristic generating means includes means for re- 
ceiving signals representative of various programs gram 



33. The method recited in claim 32 wherein the step 
of monitoring the light emanating from the display 
includes the step of monitoring the light with a photoe- 
lectric device, and indicating a detected predetermined 

5 event when the output of the photoelectric device 
changes by a predetermined amount. 

34. The method recited in claim 31 wherein the step 
of detecting predetermined events includes the step of 
determining the average amplitude of the video signal 

10 and indicating a detected predetermined event when 
the average amplitude of the video signal changes by a 
predetermined amount. 

35. The method recited in claim 31 wherein the step 
of detecting predetermined events includes the step of 

15 monitoring the video envelopes of at least one field of 
two frames. 

36. The method recited in claim 35 wherein the step 
of monitoring the video envelope of two frames m- 
cludes the step of comparing the envelopes of at least 



to be monitored and means for continuously extracting 20 one field of two consecutive frames. 
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and storing said predetermined characteristics of the 
programs being monitored to generate said reference 
predetermined characteristics. 

28. A method for identifying signals for determining 
audience ratings comprising the steps of operating on 
the signal to be identified in order to extract a feature 
string, comparing the feature string with various feature 
strings corresponding to known signals and identifying 
the signal when the feature string of the signal to be 
identified correlates with one of the feature strings cor- 30 
responding to a known signal within a predetermined 
tolerence, wherein the step of extracting the feature 
string from the signal to be identified comprises the 
steps of: 

monitoring predetermined changes in a predeter- 35 
mined parameter of the signal to be identified; 

determining the time intervals between the predeter- 
mined changes in the predetermined parameter; 
and 

utilizing the time intervals between successive prede- 40 
termined changes to generate the feature string, the 
feature string being a digital representation of a 
predefmed number of said time intervals. 

29. The method recited in claim 28 wherein the signal 

to be identified includes an audio signal, and wherein 45 
the step of extracting the feature string includes the step 
of monitoring the amplitude of the audio signal and 
determining the time intervals that the amplitude of the 
audio signal departs from a predetermined level of am- 
plitude. 

30. The method recited in claim 29 wherein the pre- 
determined level amplitude is a level that corresponds 
substantially to silence, and said time interval corre- 
sponds to the time interval between successive periods 
of silence in the audio signal. 

31. The method recited in claim 28 wherein the signal 
to be identified includes a video signal, and wherein the 
step of extracting the feature string includes the steps of 
detecting predetermined events in the video signal and 
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37. The method recited in claim 35 wherein the step 
of monitoring the video envelope of two frames in- 
cludes the step of comparing the envelopes of at least 
one field of two non-consecutive frames. 

38. The method recited in claim 37 wherein the step 
of comparing the video envelope of two non-consecu- 
tive frames includes the step of comparing the video 
envelopes of two frames spaced at approximately a 
twenty to forty frame interval. 

39. The method recited in claim 38 wherein the step 
of comparing the video envelope of two frames includes 
the step of sampling said video envelope and comparing 
counterpart samples. 

40. The method recited in claim 31 wherein the step 
of detecting predetermined events includes the steps of 
monitoring the video envelopes of two frames, obtain- 
ing a predetermined number of samples from each enve- 
lope and comparing the magnitudes of counterpart sam- 
ples to obtain a difference in counterpart samples, tak- 
ing the average of the differences, and indicating a de- 
tected predetermined event whenever the average ex- 
ceeds a predetermined level. 

41. The method recited in claim 40 wherein the step 
of comparing and taking the difference of counterpart 
samples includes the step of taking the difference of the 
absolute values of the counterpart samples. 

42. The method recited in claim 40 wherein said pre- 
determined number of samples is eighty. 

43. The method recited in claim 40 further including 
the step of obtaining a signature including the step of 
monitoring the video envelope of a predetermined 
frame, obtaining a predetermined number of samples 
from said envelope, obtaining the average of a predeter- 
mined number of consecutive samples to define said 
signature. 

44. The method recited in claim 31 wherein the step 
of detecting predetermined events includes the steps of 
determining the average level of the video signal, deter- 
mining the amplitudes of predetermined portions of the 



determining the time intervals between detected prede- 60 video signal, comparing the amplitudes of the predeter- 



termined events to generate the feature string. 

32. The method recited in claim 31 wherein the step 
of detecting predetermined events includes the step of 
monitoring the light emanating from a display on which 
the video signal is displayed, and indicating a detected 
predetermined event when the amount of light emanat- 
ing from the display changes by a predetermined 
amount. 



mined portions of the video signal with the average 
amplitude of the video signal and assigning one of a first 
and a second designation to each portion depending on 
whether the amplitude of the predetermined portion 
65 exceeds or is less than the average amplitude, utilizing 
the designations so obtained to obtain a frame code, 
comparing the frame code with the frame code of a 
previous frame, and indicating a change if the current 
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frame code differs from the previous frame code by a 
predetermined amount. 

45. The method recited in claim 28 wherein said sig- 
nal to be identified includes a video signal said video 
signal includes a color signal, wherein the step of ex- 
tracting the feature string includes the step of determin- 
ing the time interval between predetermined changes in 
the color signals. 

46. The method recited in claim 45 wherein the step 
of determining changes in the color signal includes the 
step of comparing a function of the color signal with the 
average of the color signal and determining the time 
intervals when the function of the color signal exceeds 
the average of the color signal and the time intervals 
when the function of the color signal is less than the 
average of the color signal to extract the feature string. 

47. The method recited in claim 46 wherein the func- 
tion of the color signal and the average of the color 
signal is determined for only a single line of the frame. 

48. The method recited in claim 47 wherein the single 
line is near the top of the image. 

49. In a system for identifying signals for determining 
audience ratings having means for operating on the 
signal to be identified in order to extract a feature string, 
means for comparing the feature string with various 
feature strings corresponding to known signals and for 
identifying the signal when the feature string of the 
signal to be identified correlates with one of the feature 
strings corresponding to a known signal within a prede- 
termined tolerance, the improvement wherein the fea- 
ture string extracting means comprises: 

means for monitoring predetermined changes in a 
predetermined parameter of the signal to be identi- 
fied; 

means for determining the time intervals between the 
predetermined changes in the predetermined pa- 
rameter; and 

means for utilizing the time intervals between succes- 
sive predetermined changes to generate the feature 
string, the feature string being a digital representa- 
tion of a predetermined number of said time inter- 
vals. 

50. The improvement recited in claim 49 wherein the 
signal to be identified includes an audio signal, and 
wherein said feature string extracting means includes 
means for monitoring the amplitude of the audio signal 
and determining the time intervals that the amplitude of 
the audio signal departs from a predetermined level of 
amplitude. 

51. The improvement recited in claim 50 wherein the 
predetermined level amplitude is a level that corre- 
sponds substantially to silence, and said time interval 
corresponds to the time interval between successive 
periods of silence in the audio signal. 

52. The improvement recited in claim 49 wherein the 
signal to be identified includes a video signal, and 
wherein said feature string extracting means includes 
means for detecting predetermined events in the video 
signal and determining the time intervals between de- 
tected predetermined events to generate the feature 
string. 

53. The improvement recited in claim 52 wherein 
predetermined events detecting means includes means 
for monitoring the light emanating from the display on 
which the video signal is displayed and indicating a 
scene change predetermined event when the amount of 
light emanating from the display changes by a predeter- 
mined amount. 
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54. The improvement recited in claim 53 wherein the 
display light monitoring means includes means for mon- 
itoring the light with a photoelectric device, and indi- 
cating a detected predetermined event when the output 

5 of the photoelectric device changes by a predetermined 
amount. 

55. The improvement recited in claim 52 wherein the 
predetermined event detecting means includes means 
for determining the average amplitude of the video 

10 signal and indicating a detected predetermined event 
when the average amplitude of the video signal changes 
by a predetermined amount. 

56. The improvement recited in claim 52 wherein the 
predetermined event detecting means includes means 
for comparing the video envelopes of two frames. 

57. The improvement recited in claim 56 wherein the 
video envelope monitoring means includes means for 
comparing the envelopes of at least one field of two 
consecutive frames. 

58. The improvement recited in claim 56 wherem the 
video envelope monitoring means includes means for 
comparing the envelopes of at least one field of two 
non-consecutive frames. 

2^ 59. The improvement recited in claim 58 wherein the 
video envelope comparing means includes means for 
comparing the video envelopes of two frames spaced at 
approximately twenty to forty frame intervals. 

60. The improvement recited in claim 56 wherein the 
video envelope comparing means includes means for 
sampling said video envelope and comparing counter- 
part samples. 

61. The improvement recited in claim 52 wherein said 
predetermined event detecting means includes means 

35 for monitoring the video envelopes of two frames, 
means for obtaining a predetermined number of samples 
from each envelope, means for comparing the magni- 
tudes of counterpart samples to obtain a difference in 
counterpart samples, and means for taking the average 

40 of the differences and for indicating a detected predeter- 
mined event whenever the average exceeds a predeter- 
mined level. 

62. The improvement recited in claim 61 wherein the 
comparing and counterpart samples difference taking 

45 means includes means for taking the differences of the 
absolute values of the counterpart samples. 

63. The improvement recited in claim 61 wherein said 
predetermined number of samples is eighty. 

64. The improvement recited in claim 61 further in- 
50 eluding means for obtaining a signature including means 

for monitoring the video envelope of a predetermined 
frame and obtaining a predetermined number of samples 
from said envelope, and means for obtaining the aver- 
age of a predetermined number of consecutive samples 
55 to define said signature. 

65. The improvement recited in claim 52 wherein the 
step of determining Event 2s includes means for deter- 
mining the average level of the video signal, means for 
determining the amplitudes of predetermined portions 

60 of the video signal, means for comparing the amplitudes 
of the predetermined portions of the video signal with 
the average amphtude of the video signal and assigning 
one of a first and a second designation to each portion 
depending on whether the amplitude of the predeter- 

65 mined portion exceeds or is less than the average ampli- 
tude, means for utilizing the designations so obtained to 
obtain a frame code, means for comparing the frame 
code with the frame code of a previous frame and indi- 
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eating a change if the current frame code differs from 
the previous frame code by a predetermined amount. 

66. The improvement recited in claim 50 wherein said 
signa] to be identified includes a video signal and said 
video signal includes a color signal, wherein the feature 5 
string extracting means includes means for determining 
the time interval between predetermined changes in the 
color signals. 

67. The improvement recited in claim 66 wherein the 
color signal change determining means includes means 
for comparing a function of the color signal with the 
average of the color signal and determining the time 
intervals when the function of the color signal exceeds 
the average of the color signal and the time intervals 
when the function of the color signal is less than the 
average of the color signal to extract the feature string. 

68. The improvement recited in claim 67 wherein the 
function of the color signal and the average of the color 
signal is determined for only a single line of the frame. 

69. The improvement recited in claim 68 wherein the 
single line is near the top of the image. 

70. A system for identifying signals for determining 
audience ratings comprising: 

means for detecting the occurence of predetermined 
events in the signal; 

means responsive to said detecting means for deter- 
mining the time intervals between successive occu- 
rences of said predetermined events; and 

means responsive to said time intervals for generating 
a signal identifying feature string in response to 
said time intervals, said signal identifying feature 
string being a digital representation of a predefined 
number of said time intervals. 

71. The system recited in claim 70 further including 35 
means for comparing the feature string with reference 
feature strings representative of known signals and for 
identifying said signal when said feature string corre- 
lates with one of said reference feature strings within a 
predetermined coefficient of correlation. ^ 

72. A method of identifying a television signal re- 
ceived from either a television receiver or a video re- 
corder for determining audience viewing habits com- 
prising the steps of: 

detecting predetermined operational modes of the 45 

audience member's television receiver and the 

video recorder, 
detecting the occurence of a first event in one of the 

operational modes; 
detecting the occurence of a second event in the so 

signal to be identified; 
extracting a signature from a single frame of the video 

signal to be identified after the occurence of the 

second event; and 
storing the extracted signature, the time of the signa- 55 

ture extraction and the detected operational mode. 

73. The method recited in claim 72 further including 
the step of comparing the extracted signature with ref- 
erence signatures. 

74. The method recited in claim 73 wherein said ref- 60 
erence signatures are extracted by the steps of: 

monitoring a known signal; 

detecting the occurrence of said second event in the 
known signal; 

extracting a reference signature from the known sig- 65 
nal after the occurence of a second event; and 

storing the reference signature and the time of occu- 
rence thereof. 
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75. The method recited in claim 74 wherein the step 
of comparing includes the step of comparing the ex- 
tracted signature with reference signatures that oc- 
curred at approximately the same time as the extracted 
signature. 

76. The method recited in claim 72 wherein said sec- 
ond event is a scene change from one video scene to 
another video scene. 

77. The method recited in claim 72 whrein said televi- 
son signal is a color television video signal and said 
second event is a change in the color of at least a portion 
of successive frames. 

78. The method recited in claim 72 wherein said sec- 
ond event is a still picture. 

79. The method recited in claim 72 wherein said sec- 
ond event is a predetermined change in the magnitude 
in an audio signal of the television signal. 

80. The method recited in claim 72 wherein said first 
event is the turn-on of the television receiver. 

81. The method recited in claim 72 wherein said first 
event is a change in the channel to which the television 
signal receiver is tuned. 

82. The method recited in claim 72 wherein said first 
event is the elapse of a predetermined length of time 
after the occurence of the previous first event. 

83. The method recited in claim 82 wherein said pre- 
determined length of time is approximately 5 to 10 min- 
utes. 

84. The method recited in claim 72 wherein said step 
of extracting said signature includes the step of low pass 
filtering a video signal of the television signal to provide 
a video envelop signal, periodically sampling said video 
envelope signal to provide a series of video samples, and 
digitizing said video samples to provide said signature. 

85. The method recited in claim 84 wherein said step 
of low pass filtering said video signal includes the step 
of passing the video signal through a low pass filter 
having a cut-off frequency of approximately 2 to 6 kHz. 

86. The method recited in claim 85 wherein the step 
of sampling said video envelope signal includes the step 
of taking eighty samples of said video envelope signal 
and taking the average of every eight consecutive sam- 
ples to generate ten features that comprise the feature 
string. 

87. The method recited in claim 85 wherein the step 
of sampling said video envelope signal includes the step 
of taking a predetermined number of samples of said 
video envelope signal and taking the averages of prede- 
termined consecutive samples to generate a plurality 01 
average signals, each average signal defining one of the 
features that comprises the feature string. 

88. A system for identifiying program material 
viewed on a television receiver or recorded by a video 
recorder for determining audience viewing habits com- 
prising: 

means for detecting predetermined operational 
modes of the television receiver and the video 
recorder, 

means for detecting a first predetermined event in one 
of the operational modes; 

means for detecting a second predetermined event in 
the program material; 

means responsive to the detected predetermined op- 
erational modes for extracting a predetermined 
characteristic of the program material from a single 
frame of a video signal after the occurrence of said 
second predetermined event; and 
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means for storing the predetermined characteristic, 98. A system as recited in claim 92 wherein said first 

the time the predetermined characteristic is ex- event detecting -means includes means for detecting a 

tracted and the detected predetermined operational change in the channel to which the television signal 

mode. receiver is tuned. 

89. A system as recited in claim 88 wherein said ex* S 99. A system as recited in claim 92 wherein said first 
tracting means includes means for extracting said prede- event detecting means includes means for detecting the 
termined characteristic upon the second occurence of elapse of a predetermined length of time after the occur- 
the second predetermined event foU owing the occu- rence of the previous first event. 

rence of said first event. 100. A system as recited in claim 99 wherein said 

90. A system as recited in claim 89 wherein said ex- 1^ detecting means includes means for setting the predeter- 
tracting means includes means responsive to a detected mined length of time to approximately 5 to 10 minutes, 
recording operational mode for extracting said prede- 101. A system as recited in claim 92 wherein said 
termined characeristic upon each occurrence of the means for extracting said signature includes means for 
second predetermined event* and wherein the predeter* low pass filtering a video signal of the television signal 
mined characteristic further includes the time interval to provide a video envelope signal, means for periodi- 
between the second event and the immediately preced- cally sampling said video envelope signal to provide a 
ing second event. series of video samples, and means for digitizing said 

91. A system as recited in claim 88 further including video samples to provide said signature. 

means for comparing the extracted predetermined char- 102. A system as recited in claim 101 wherein said 

acteristics with stored reference predetermined charac- low pass filtering means has a cut-off on the order fre- 

teristics to identify the program material. quency of approximately 2 to 6 kHz. 

92. A system for identifying a television signal re- 103. A system as recited in claim 102 wherein said 
ceived from both a television receiver and a video re- video envelope signal sampling means includes means 
corder for determining audience viewing habits com- for taking approximately eighty samples of said video 
prising: envelope signal and means for taking the average of 

means for detecting predetermined changes in the every eight consecutive samples to generate ten features 
operational modes of the television receiver and that comprise the feature string, 
the video recorder; 104. A system as recited in claim 101 wherein said 
means for detecting the occurence of a first event in video envelope signal sampling means includes means 
one of the operational modes; for taking a predetermined number of samples of said 
means responsive to both the detection of a first event video envelope signal and means for taking the averages 
and a detected viewing operational mode for de- of predetermined consecutive samples to generate a 
tecting the occurrence of a second event in the plurality of average signals, each average signal defin- 
signal to be identified; 35 ing one of the features that comprises the feature string, 
means responsive to a detected recording operational 105. In a system for identifying programs viewed on 
mode for detecting the occurence of a second a television receiver, wherein the programs are ob- 
event in the signal to be identified; tained from a video recorder and from an external pro- 
means responsive to the detection of a second event gram source, the improvement comprising: 
for extracting a signature from a single frame of the 4Q means responsive to the on/off condition of the tele- 
video signal to be identified after the occurence of vision; 

the second event; and means responsive to the on/off condition and the 

means for storing the extracted signature and the play/record condition of the video recorder; 

detected operational mode. logic means responsive to the television condition 

93. A system as recited in claim 92 further including 45 responsive means and the video recorder condition 
means for comparing said stored signatures with refer- responsive means for providing an indication if the 
ence signatures obtained from known signals to identify program being viewed is being obtained from the 
the signal. video recorder or from the external program 

94. A system as recited in claim 93 further including source; 

means for obtaining the reference signatures, said refer- 50 means for identifying the program being viewed, said 

ence signal obtaining means including means for moni- identifying means being responsive to said logic 

toring known signals, means for detecting the occur- means and operative to identify said program ac- 

rence of second events in said known signals, and means cording to two different criteria depending on 

responsive to the detection of the second events for whether the program being viewed is obtained 

extracting reference signatures from the known signals 55 from the external program source or form the 

because real time data is not necessary to make the video recorder. 

identification. 106. The improvement recited in claim 105 further 

95. A system as recited in claim 94 wherein said com- including means responsive to said video recorder con- 
paring means includes means for comparing an ex- dition responsive means for storing data representative 
tracted signature with reference signatures that oc- 60 of the program being viewed in two different manners 
curred at approximately the same time. depending on whether or not the program being viewed 

96. The system reicted in claim 92 wherein said sec- is being recorded. 

ond event detecting means includes means for detecting 107. In a system for determining the viewing habits of 

a scene change from one video scene to another video a household having a video recorder capable of record- 
scene. 65 ing programs obtained from an external program source 

97. A system as recited in claim 93 wherein said and playing video programs, and a television receiver 
means for detecting said first event includes means for for viewing programs obtained from the external pro- 
detecting the turn-on of the television receiver. gram source and from the video recorder, said system 
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having a home unit and a central unit, the improvement 
comprising: 

means responsive to predetermined operating condi- 
tions of said television receiver and said video re- 
corder to provide signals representative of televi- 5 
sion viewing, recording, and playback; 

means for detecting during viewing a first predeter- 
mined event in the program material; 

means responsive to the detection of said first prede- 
termined event for detecting during viewing a '0 
second predetermined event in the program mate- 
rial; 

means responsive to the detection of said second 
predetermined event for extracting during viewing 
at least one predetermined characteristic of the 
program material; 

means for storing at the home unit the predetermined 
characteristic of the viewed program material and 
the time that the predetermined characteristic is 
extracted; 

means for continuously detecting during recording 
the second predetermined events in the program 
material; 

means responsive to the detection of said second 
predetermined events for extracting during record- 
ing at least one predetermined characteristic of the 
program material; 

means for storing during recording the predeter- 
mined characteristics and the times that the prede- 
termined characteristics are extracted; 

means for comparing the predetermined characteris- 
tics stored during viewing and recording and their 
times of occurrence vAxh reference predetermined 
characteristics stored in the central unit that oc- 35 
curred at substantially the same time to identify the 
programs viewed and recorded; 

means for detecting during playback the first prede- 
termined event in the program material; 

means responsive to the detection of said first prede- 40 
termined event for detecting during playback the 
second predetermined event in the program mate- 
rial; 

means responsive to the detection of said second 
predetermined event for extracting during play- 45 
back at least one predetermined characteristic of 
the program material; 

means for recording during playback the predeter- 
mined characteristic; and 

means for comparing the predetermined characteris- 50 
tic recorded during playback with reference prede- 
termined characteristics stored at the central unit at 
the time of recording to identify the program. 
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108. In a system for determining the viewing habits of 
a household having a video recorder and a television 
receiver, said system having a home unit and a centraJ 
unit, the improvement comprising: 
means responsive to predetermined parameters of 
said television receiver and said video recorder to 
provide signals representative of television view- 
ing, playback and recording; 
means for detecting during viewing a first predeter- 
mined event in the program material; 
means responsive to the detection of said first prede- 
termined event for detecting during viewing a 
second predetermined event in the program mate- 
rial; 

means responsive to the detection of said second 
prediermined event for extracting during viewing 
at least one predetermined characteristic of the 
program material; 
means at the home unit for storing during viewing the 
predetermined characteristic and the time that the 
predetemined characteristic is extracted; 
means for continuously detecting during recording 
the second predetermined events in the program 
material; 

means responsive to the detection of said second 
predetermined events for extracting during record- 
ing at least one predetermined characteristic of the 
program material; 
means at the home unit for storing during recording 
the predetermined characteristics and the limes of 
occurrence thereof; 
means at the home unit for determining time intervals 
between the detached predetermined second 
events during recording; 
means for detecting during playback the second pre- 
determined events in the program material; 
means responsive to the detection of said second 
predetermined events for extracting during play- 
back at least one predetermined characteristic of 
the program material; 
means at the home unit for determining time intervals 
between the detected predetermined second events 
during playback; 
means at the home unit for comparing the time inter- 
vals between the detected events during playback 
and the time intervals between the events stored 
during recording; and 
means responsive to a match provided by the com- 
paring means for comparing the predetermined 
characteristics extracted during recording with 
reference characteristics stored in the centraJ unit 

to identify the program. 

* * * * 4 
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